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OPTIMAL CONTROL USING IMBEDDING 


OF THE TERMINAL CONDITIONS* 

By Raymond C. Montgomery 
Langley Research Center 

SUMMARY 

Imbedding theory is used to construct solutions of autonomous optimal control 
problems with free termination time; this is accomplished by imbedding the trajectory 
optimization problem of interest in a continuous family of problems which are parameter- 
ized by terminal conditions. The imbedding family used is quite arbitrary but it must 
contain the original problem as one member and a problem that possesses a known solu- 
tion as another member. 

Necessary conditions are derived which determine the modifications in the control 
functions required when passing from the solution corresponding to one member of the 
family to that corresponding to another member when their terminal conditions differ 
infinitesimally. By continuously collapsing the terminal conditions of the family of prob- 
lems onto those of the original problem while appropriately modifying the control func- 
tion, the solution to the original optimization problem can be obtained. 

One important reason for using imbedding theory to solve optimization problems is 
its utility in solving singular control problems. This utility is illustrated with an example. 
A problem of finding the time -optimal maneuvers for an aerial attack where maneuvers 
are restricted to a horizontal plane is solved. The model of the airplane uses a square- 
law drag term. The controls used are airplane thrust and turning acceleration. Both 
controls are limited. The airplane maneuvers in such a way that the target vehicle 
(assumed to be nonmaneuvering) is placed in a situation favorable to deployment of the 
airplane armament in the shortest possible time. The solutions to this problem are com- 
posed of both singular and nonsingular subarcs of the trajectory. Thus, the utility of 
imbedding in solving singular optimal control problems is illustrated. 




*Part of the information presented herein was included in a thesis entitled 
"Trajectory Optimization by Terminal Imbedding” submitted in partial fulfillment of the 
requirements for the degree of Doctor of Philosophy in Aerospace Engineering, Virginia 
Polytechnic Institute, Blacksburg, Virginia, May 1969. 



INTRODUCTION 


Applications of optimization techniques in aeronautics and astronautics have 
appeared as early as the 17th century. Indeed Sir Isaac Newton used a variational 
approach to study the problem of finding minimum drag shapes for a body of revolution 
which was submersed in an extremely high-speed fluid flow at zero angle of attack. (See 
ref. 1.) For this problem the body had a given length and base diameter. More recently 
the rising interest in problems related to atmospheric flight mechanics has acted as a 

catalyst in advancing the development and application of optimal control theory. One 

> 

example of this is the defining of optimal guidance and control for the powered flight of a 
missile cpnsidered by Duersch (ref. 2). Others have studied the minimum time to climb 
problem for airplanes. (For example, see ref. 3.) This later research led to significant 
improvements in guidance for achieving a given altitude in the shortest possible time. 

The optimization problem considered in this paper may be formulated as follows: 
"Given a deterministic system with a state x(t), governed by a set of first-order differ- 
ential equations; find the control function u(t) which satisfies the given control con- 
straints, such that at some final time the state satisfies prescribed terminal constraints 
and renders a given integral cost function a minimum." 

The stated optimization problem has many aerospace applications. As formulated, 
there are four popular methods for extracting optimal controls: the calculus of varia- 
tions, the maximum principle of Pontryagin, dynamic programing, and direct iterative 
methods. Each of these four methods has proved to be a strong tool for obtaining optimal 
controls although each possesses inherent difficulties. Convergence of direct iterative 
methods is often a problem. Computational implementation of dynamic programing is 
sometimes prohibitive due to excessive memory requirements. Both the calculus of 
variations and the maximum principle suffer from difficulties inherent in multipoint 
boundary value problems as well as singular control problems. 

Ideally, the application of the maximum principle leads to a two-point boundary 
value problem (ref. 4) which may be solved by a method, introduced by Bellman (ref. 5), 
called invariant imbedding. However, certain problems may arise in which the conditions 
set down by the maximum principle fail to restrict the control functions sufficiently to 
enable a unique optimal control to be extracted. If only a finite number of control func- 
tions satisfy the conditions of the maximum principle, a direct comparison of the "cost" 
associated with each control serves to eliminate all but the optimal control. If, on the 
contrary, there is an infinite selection of controls which satisfy the maximum principle, 
then the optiriiization problem is not reducible to a simple two-point boundary-value prob- 
lem and the concept of invariant imbedding will not define the optimal control. 
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Problems possessing this difficulty have been studied by Rozonoer (ref. 6) who 
referred to them as singular control problems and by Johnson and Gibson (ref. 7). In 
reference 7 is outlined a systematic procedure for obtaining singular controls. However, 
he does not say whether the arc derived from the singular control is a portion of the 
optimal trajectory. An illustration of the approach of Johnson and Gibson applied to 
solving a singular control problem is presented in appendix A. Rozonoer has shown, for 
restricted cases where the Hamiltonian used in the maximum principle as well as the 
canonical equations are all linear in the control, that the conditions of the maximum prin- 
ciple are both necessary and sufficient to define the optimal control. However, the 
general problem of singular control is unanswered and the problem of piecing together 
optimal trajectories that contain singular controls is somewhat more complex than has 
been resolved by present control theory. 

The primary purpose of this report is to develop a field theory for optimal control 
problems that more effectively deals with the problem of piecing together singular and 
nonsingular subarcs to construct optimal trajectories. This field theory is called 
"terminal imbedding" since optimal trajectories are determined by imbedding the given 
optimization problem of interest in a family of optimization problems parameterized by 
their termination conditions. This terminal imbedding is in contrast with "general 
imbedding" (ref. 8) in which the imbedding is accomplished by considering a family of 
optimization problems parameterized by different cost functions and with dynamic pro- 
graming in which surfaces of constant optimal cost can be considered as the imbedding 
parameterization. Note, however, that with dynamic programing, the surfaces of constant 
optimal cost are not known a priori, whereas in terminal imbedding the imbedding termi- 
nation surfaces are selected a priori. The closest analogy to other methods of extracting 
optimal trajectories is invariant imbedding in which a terminal manifold parameteriza- 
tion is made holding a boundary condition constant for the purpose of solving a two-point 
boundary value problem. In the case of terminal imbedding, the terminal manifold 
parameterization is made obtaining a minimum cost for each member of the family of 
solutions. 

The basic concept underlying terminal imbedding was derived from a synthesis 
procedure for determining the feedback gains for a linear feedback control system that 
result in arbitrarily specified closed-loop pole -zero properties. The basic tool used in 
that synthesis procedure, referred to herein as the method of conversion to differential 
form, was proposed for solving algebraic equations by Yakovlev (ref. 9) and was extended, 
without vigorous analytical justification, to the synthesis of linear feedback control sys- 
tems by Montgomery and Hatch (ref. 10). Since the method of terminal imbedding relies 
heavily on the method of conversion to differential form, first a strong analytical base is 
provided for the method of conversion to differential form. Next, the general theory of 
terminal imbedding, as applied to trajectory optimization problems, is developed. Then, 
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singularities that arise in the analysis which must be considered in the practical applica- 
tion of terminal imbedding are analyzed. Finally, several applications of terminal 
imbedding are presented, among which are linear and nonlinear optimization problems, 
and singular control problems. 

SYMBOLS 

A(t,o) matrix defined by equation (10) 

Ajj matrix defined by equation (30) 

a^,...,a 0 constants defined by equation ( 66 ) 

B(t,o) matrix defined by equation (11) 

c(x^,u^,o) matrix function defined by equation (18) 

c(o) vector function of cr 

c° c(cr) evaluated at cr = 0 

c* c(o) evaluated at a = 1 

d scalar variable 

dX/da vector of Lagrange multipliers 

F(x) matrix function of x defined by equation (5) 

f(x^,ct) matrix function of and a defined by equation (20) 

F(x,i//) coefficient of the control u(t,o) in H(x,i/qu) 
f(x) algebraic vector function 

f(x,u) the state derivative as a function of x(t,o) and u(t,o) 

G(x) = V x f(x) vjf(x) 

G^(xf,uf,cr) scalar function defined by equation (19) 

G 2 (x^,u^,a) total derivative of Gj with respect to a 
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H(x,^,u) 

h(t,o) 

hi 

I 

i(x, ifd 

j(x f ) 

j(x*,cr) 

J[‘] 

K(t) 

Kl 

k 2 

M(t,o) 

m 

N 

n 

P 

R 

r 

r* 

s(x°,r) 

Si(a) 


Hamiltonian function 
vector function defined by equation (Bl) 
vector defined by equation (39) 
identity matrix 

part of H(x,i//,u) independent of control 
scalar function describing terminal manifold 
imbedded scalar function describing terminal manifolds 
functional defined by equation (B3) 

number of discontinuities in control over interval J”t 0 ,t) 

maximum normal acceleration 

constant depending on airplane drag 

vector function of t,cr 

dimension of control vector u 

number of discontinuities in ax/at occurring in time interval 
dimension of state vector x 
scalar cost function 

distance from airplane to center of termination zone 

radial distance 

radius of termination zone 

hypersphere of radius r centered at x° 

time at which ith discontinuity occurs 



t 


time 


t 0 initial time 

u(t,o) control function 

u (o) terminal control 

airplane control of normal acceleration 
V airplane speed , 

V-j. target speed 

V c airplane speed control 

V m ax maximum airplane speed 

Vmin minimum airplane speed 

v vector 

v(cr) vector of independent variables 

x(t,o) state vector with x n+ ^ being cost function 

x° x(o) evaluated at a = 0 or initial state 

x* x(o) evaluated at a = 1 

x A ,y A coordinates of position of airplane in horizontal plane 
x^ position of harmonic oscillator 

a (xf) function in equation (57) 

lower control limit on Uj 
j3(x^) function in equation (57) 

upper control limit on Uj 
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?V y 2 


Af i (o) 


Au^ 

6(x f ) 


e 


e 

6 


M 


Si 


$(t,t 0 ,a) 

*U 

x(t»o) 

P 

if/ 1 

a 


v 

v« 


functions independent of speed control 
change in f(x,u) across ith discontinuity 
change in u across ith discontinuity 
function defined by equation (59) 
is an element of 
positive number 

airplane orientation in horizontal plane 
(n+1) -dimensional unit vector with M n+ i unity 
region of t,a space free of discontinuities 
imbedding parameter 

matrix function defined by equations (13) and (14) 
element of ith row and jth column of 4> 
vector function 
costate variables 

column vector whose components are first n components of vector i// 
control space 
restricted control space 

matrix whose element of ith row and jth column is af^auj 

matrix whose i,j element is ^i/^jj 

matrix whose element of ith row and jth column is afj y'oKj 
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V X H 


QTT 

column vector whose ith element is ~P-(x,\(s,u) 

9xi 


VjjJ column vector whose ith element is |^(x^) 

V x j(x^,ct) column vector whose ith element is , a) 

V xxJ 


V 


S 2 J 


matrix whose element of ith row and ith column is Sx, 

8Xj J 

QTT 

column vector whose ith element is — — (x,t2/,u) 

9^i 


{x|P} set x satisfying property P 

A equality by definition 


• norm 


range of a real variable x such that a g x < b 

a Riemann integral taken over the interval £ 

A dot indicates differentiation with respect to time. A superscript T indicates 
a transpose whereas a subscript T indicates target. Superscript and subscript f 
indicate that the function is evaluated at termination time. 



IMBEDDING FOR SYSTEMS OF ALGEBRAIC EQUATIONS 

The recent development of high-speed digital computers has created a tool which 
was not previously available to aerospace scientists. Problems which heretofore have 
been intractable due to immense computational requirements can now be considered. One 
example of such is the analysis and design of high-order linear feedback control systems. 
Much of the early effort in this area was devoted to the development of simplified methods 
of analysis which could analyze the stability and control characteristics of single-input, 
single -output systems. 

Generally the synthesis of multiple input feedback control systems leads to sets of 
nonlinear algebraic equations which must be solved for feedback gains. The modern 
high-speed computer enables one to apply iterative approaches to the solving of algebraic 
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equations which were previously undesirable. Newton's procedure is a popular method 
for obtaining solutions of algebraic equations (ref. 11). The convergence of Newton's 
method, as with most iterative methods, is not usually known a priori. This difficulty 
with convergence can sometimes be avoided by using imbedding theory to obtain the solu- 
tion desired. This is accomplished by considering the solution of the algebraic problem 
of interest as one member of a continuous family of solutions wherein one member of the 
family is known. Then conditions are derived that allow determination of one member of 
the family given a neighboring member. This concept of imbedding when applied to the 
solution of algebraic problems will be referred to as the method of conversion to differen- 
tial form, for reasons which will be evident later. 

As an aid in explaining the method of conversion to differential form for solving 
nonlinear algebraic equations consider the scalar equation 

f(x) = x 2 - 3x + 2 = c (1) 

A graph of f(x) against x satisfying equation (1) is presented in figure 1. To solve 
the equation f(x) = 0, one should first let both c and x in equation (1) be functions of 
a dummy variable a which varies from 0 and 1. That is, let c 4 c(it) and x 4 x(a). 

The basic idea is to arbitrarily select an initial point, say x° = x(0) and a variation 
c(ct) such that c(cr) passes through the point c(0) = f(x°) and the desired point c(l) = 0. 



A typical curve c(a) satisfying these requirements is the straight-line segment illus- 
trated in figure 2. For that graph the value of c(0) was calculated by substituting 
x(0) = x° = 0 into equation (1). Then, the variation in x(o) required by the identity 
f(x(o)) = c(cr) is determined by using a differential form of equation (1); namely, 


fw”” > - gW 
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For this particular example, using the linear variation for c(o) as indicated in figure 2 
where dc/dcr has been made equal to -2, the derivative dx/da must satisfy the equation 

*E(o) = — — — 

dcr ’ 2x - 3 

This equation can be integrated (numerically) over the interval 0 = a = 1 with an initial 
condition x(0) = 0. The value of x at a = 1 satisfies the identity f(x(l)) = c(l) = 0 
and a solution of the equation f(x) = 0 is obtained. Note that if an initial value of x = 3 
was selected, the solution obtained would be x = 2 and not the one indicated in figure 2, 
which is x = 1. This example illustrates the nonuniqueness of solutions of algebraic 
equations and the fact that the particular solution obtained by using imbedding is depen- 
dent on the imbedding parameterization selected, for example, the choice of x° and 
c(o). 

For this simple example, it is obviously easier to solve the nonlinear algebraic 
equation using the quadratic formula than to use the method of conversion to differential 
form. However, for complex sets of nonlinear algebraic equations such as those arising 
in the synthesis of linear feedback control systems no general closed-form solutions are 
available; however, the method of conversion to differential form does remain applicable. 

Assume that one wishes to solve the nonlinear set of algebraic equations f(x) = c 
for a vector x, say x^, which leads to a specific vector c, say c 1 . Stipulate that n, 
the dimension of x, is greater than or equal to m, the dimension of c. Let c and x 
be functions of the imbedding parameter a, and select c(cr) such that it satisfies 
c(0) = c°. Hence, 

c° ^ f( x o) ( 2 ) 

and is calculated from an arbitrary x°. Other requirements on c(o) are that 
c(l) = cl and c(o) is differentiable on the interval 0 £ a i 1. The linear function 

c(cr) = c° + a(c* - c°) 

is one such example. The variation in x(cr) - required by the identity f(x(cr)) = c(o) - 
must satisfy the relation 

v x f(x( 0 )) ^(o) = ~i°) (3) 

da da 

Equation (3) is a set of implicit differential equations that are linear in dx/da, and which 
usually can be integrated numerically once the function £j£(a) over the interval 

0 = a ^ 1 and the initial conditions x(0) = x° are given. When the dimension of x is 
greater than that of c the integration process is usually nonunique. The function ^(a) 
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can be thought of as a forcing function which forces x(cr) to follow some path which 
preserves the identity c(o) = f(x(cr)). Hence, the value of x(o), at a = 1, should satisfy 
the equation f(x(l)) = c*. 

Since equation (3) is an implicit set of differential equations one does not know, 
a priori, whether the solution x(cr) can be continued from cr = 0 to cr = 1. Therefore, 
it is of interest to determine the extent to which solutions of x(a) can be continued in 
terms of restrictions on both V x f(x) and c(o). Let x and c satisfy 

c = f(x) (4) 

and let x(0) = x° and c(0) = c° satisfy equation (2). Assume that the matrix V x f(x) 
is of rank m < n for vectors x e S(x°,r) where S(x°,r) is the set of vectors x such 
that || x - x°|| = r. In S(x°,r) equation (3) has an n - m parameter family of solu- 
tions for dx/dcr. The particular member of this family of solutions for dx/dcr which 

1 dx^ dx 

minimizes the function — — — tends to minimize the rate at which the solution x(o) 

2 da da w 

leaves the sphere S(x°,r) for any given c(a) variation. Hence, consider the solution 

1 Hy'! dx 

of equation (3) which minimizes ^ subject to the constraint of equation (3). This 

problem is considered in reference 12. Only the essential results of the' analysis will be 
presented here. To incorporate the constraint, construct an alternate function 


n A 1 dx T dx , dX T 

- 2‘d^c^ + ~d7 




dc 

da 


to be minimized where dx/da is an m dimensional set of Lagrange multipliers 
appending the constraints to the original cost function. The solution to the optimization 
problem posed must satisfy 


where 


F(x) 


dx 


0 

da 


dX 


dc 

da 


da 


F(x) ^ 


I 

Vjf(x) 

V x f(x) 

0 


(5) 


Since V x f(x) is of rank m for x e S(x°,r), 
x € S(x°,r) whose inverse is 


F(x) is a nonsingular matrix for 
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where 



vJfG-Wxf 

i V^fG -1 
1 ^ 

I 

G- X V x f 

1 

O 


G = G(x) ^ v x f(x) vjf(x) 


Note that G(x) is nonsingular since V x f(x) is of rank m for any x e S(x°,r). Next, 
define a function r(d) such that r is the largest radius of the hypersphere S(x°,r) 

with ||f“ 1 (x)|| £ d for all x e S(x°,r). Then one can see that 


dx 

da 


dx 

da 

dX 

da 



dc 

c H 

dc 

da 

= u 

da 


If m = n then ||dx/da|j i jj V“ 1 f(x)|| • || dc/daj|. In this case, define r(d) as the 
largest r such that 


V^x)!] gd, x e S(x°,r). 


Now, since 



SO) d /3 



if one requires that 


r(d) ^ d 




then for some value of d, |[x(a) - x°|| i r if 0 = al 1, and thus x(a) is contained 
in S(x°,r). The method of conversion to differential form can therefore be successfully 
applied, and the following sufficient condition has been demonstrated. 


Theorem 1 : Consider equation (4) with the condition min and given point x° 
and function c(a) which is differentiable on the interval 0 i a i 1 and satisfies 
c(0) = f(x°). Then if m < n, let V x f(x) be of rank m, x e S(x°,r(d)) where r(d) is 


the largest r for which 
maximum r for which 


|jF -1 (x) 

v^fM 


id, x € S(x°,r); if m = n, define r(d) as the 
id, x e S(x°,r). If for some value of d, 
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then there exists a solution x(cr) of equation (4) such that 


and 


f(x(cr)) = c(cr) 


w|- 


dc 

do 


On the interval 0 Sag 1, the rank of V x f(x) is m. 

To illustrate the use of this theorem, consider the example noted as equation (1) 
except that c has been replaced with c(0) = c°. The gradient matrix is V x f = 2x - 3, 

^ d to give 


and taking x(0) = 0, the function r(d) can be calculated from 
3d - 1 


V _1 f 
v x 1 


r(d) = 


2d 


where d = If one designates c(o) to be linear and of the form 
o 


c(o) = c°(l - d) 


then 


I, 


$£jr) 

0 II ^ 


d/3 


- I c° 


Consequently, the conditions of theorem 1 imply that a solution of the equation 


x - 3x + c° = 0 


( 6 ) 


exists provided that r(d)=jc°]d for some d. This condition reduces to 

3d - ~ 1 s I c°|d 
2d 1 1 

This condition is graphically illustrated in figure 3 where the function r(d) is plotted 
against d; note that if the straight line passing through the origin, with a slope of | c° ( , 
intersects the curve r(d) a solution to equation (6) is assured and can be obtained by 
the method of conversion to differential form. Also, from figure 3 it is apparent that 
solutions cannot be assured for all | c° |. Indeed, if | c° | > 9/8, theorem 1 cannot be 
applied even though solutions of equation (6) are known to exist. This example empha- 
sizes the fact that theorem 1 is only a sufficient condition and is not generally a neces- 
sary one. 




Figure 3.- Graph of r(d) against d which illustrates 
the applicability of theorem 1. 

The method of conversion to differential form has been applied to the design of 
multiaxis stability augmentation systems for aerospace vehicles in reference 10. 

TERMINAL IMBEDDING IN OPTIMAL CONTROL THEORY 

In this section the method of terminal imbedding is applied to trajectory optimiza- 
tion problems. The problem to which the method of terminal imbedding is directed is 
precisely stated; the general concept, underlying the terminal imbedding, is outlined; 
geometric implications and assumptions of the theory are discussed; and the principle 
result of the analysis is presented. 

Statement of the Problem 

The method of terminal imbedding is directed to a slightly restricted form of the 
general optimization problem mentioned in the introduction. Let a control process be 
given which is governed by the autonomous differential equation 

x = f(x,u) (7) 

where x is an (n+1) -dimensional state vector and u is an m-dimensional control 
vector. The control vector u is assumed to be constrained to some manifold region 
of an m-dimensional Euclidean space fi. A manifold region is defined as a region of 
Euclidean space such that u € 12 implies that each component of u satisfies the 
inequalities S Uj S /3j for i=l,2,...,m and either finite or infinite. The 

function f(x,u) is assumed to have first partial derivatives with respect to the com- 
ponents of the x and u vectors. 


14 



Suppose that there is given a simply connected (n-1) -dimensional space called the 
terminal manifold. Let it be described by the set of vectors x satisfying the equality 
J(x) = 0. The function J(x) is a scalar function of x, taken to be independent of x n+ p 
having V xx J(x) defined at each boundary point of the terminal manifold. An admissible 
trajectory is defined to be a continuous and piecewise differentiable vector function x(t) 
defined over the interval t 0 ^ t ^ tf of the independent variable t which satisfies 

(1) x(t 0 ) = x° with x° +1 = -0 

(2) J(x(t)) > 0 for t Q ^ t < tf 

(3) J(x(tf))=0 

(4) the governing differential equation (7) on each differentiable subarc of x(t) for 
some piecewise continuous control function u(t) e £2 for t 0 = t ^ tf 

The control functions u(t) corresponding to admissible trajectories are called admis- 
sible controls. The problem considered in this section is to find an admissible control 
u(t) which renders x n+ ^(tf) a minimum. 

The Concept of Terminal Imbedding 

In attacking the stated optimization problem using the method of terminal imbedding 
the original problem is considered to be imbedded in a family of optimization problems, 
transferring the vector x from the initial point x° to each of a family of terminal 
manifolds which are parameterized by a scalar cr. The basic concept underlying the 
method of terminal imbedding is "with a knowledge of the control function and corre- 
sponding motion trajectory, which will optimally transfer the state x, from the initial 
point x° to a surface described by j(x^,a) = 0, find the modifications required to trans- 
fer the state from x° to the surface described by j(x^,cr+e) = 0 optimally for e > 0." 
Continued application of this idea will allow one to construct the control function and 
corresponding trajectory of motion required to transfer the state from x° to the surface 
described by j(x^) = 0 in an optimal manner. Requirements on the parameterized family 
of terminal manifolds are such that the stated optimization problem has a known solution 
or one easily obtained for one member of the family of imbedded manifolds corresponding 
to a value of a, say, a = 0, and that the family of terminal manifolds includes the termi- 
nal manifold of the original problem statement for some value of a, say, ct = Of. The 
parameterized family of terminal manifolds is described by the scalar function J(x,o) 
which is identical to J (x) for a = Of . 

The parameterization of the terminal manifolds is not generally unique. One pos- 
sible choice is the family described by 

J(x,cr) = J(x) + J(x°)(a - 1) 
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where J(x) is the function describing the terminal manifold of the original problem. 
Here, x° is the initial condition of the original problem, and Of is assumed to be 1. 
This family satisfies both of the stated conditions. At a = 0 the initial condition satis- 
fies J(x°,0) = 0 and the optimal trajectory is known; at cr=l the function J(x,l) 
reduces to J(x) which describes the original terminal manifold. 

The stated optimization problem is therefore imbedded in an initial value problem. 
The schematic diagram at the top of figure 4 illustrates the evolution of the optimal 
trajectories for various values of a. Identify a particular optimal trajectory x(t) and 
the associated optimal control u(t), that correspond to a particular value of a as x(t,cr) 
and u(t,a), respectively. In the analysis which follows, the continuity of the vector x(t,cr) 
in both t and a is required. Because of the governing differential equations, con- 
tinuity in t is assured; however, continuity in a is not. Furthermore it is assumed 
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that x(t ,o) has continuous partial derivatives in t and a on each interval, ^ for 
i = 1,2,...,N+1, where 

1 1 0 < t < S 1 (a); 0 ^ a S cr f j 
ii : jjt,a)| S^o) < t < Si(a); 0 S a S a f J 

and 

%+l : jjM | S N (a) < 1 < 0 = CT = °fj 

The values of Si(o),...,Sj^(o) represent values of t for which the function x(t,a) may 
have discontinuous partial derivatives in t. These times are not known a priori and 
must be determined. They normally represent points of discontinuity in the control func- 
tion u(t,a). 

The lower sketch of figure 4 graphically illustrates a typical variation of the kth 
component of x, say, x k with both t and a. Note that the locus of discontinuities in 
9xk 

-^-(t,a) is indicated by the curves S-^o) and S 2 (o'). The terminal time t f in gen- 
eral depends on a; this is indicated by tf(a) in the sketch. At a = cr f the function 
x k (t,o-f) is the desired optimal trajectory. The functions Si(cr),...,S N (o) and u(t,cr) are 
assumed to be bounded and differentiable functions of a over their region of definition. 

Theoretical Results 

The principal result .- From the concept of terminal imbedding, the state x is 
considered a function of two scalar variables t and a. According to equation (7), the 
solution x(t,a) must satisfy the equation 

-|r(t,cr) = f(x(t,o),u(t,o)) (8) 

on every interval hence, 

p t 

x(t,o) = x° + \ f(x(r,a),u(r,a)) dT 

Because of the requirement that x(t,cr) and u(t,cr) must be differentiable in both t 
and a on intervals ^ and that the functions Si(o) be differentiable, it follows that 
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for t,a in In equation (9), K(t) represents the number of Sj(a) functions con- 

tained in the interval |t 0 ,t) and the following definitions have been used 

A(t,cr) A v x f(x(t,a),u(t,a)) (10) 


B(t,o) ^ V u f(x(t,o),u(t,o)) 


and 


Afi(a) A Urn f(x(S i ,a),u(S i -e ,a)) - f (x(Si,a),u(Si+e,a))J 


e>0 


(11) 


Equation (9) is a linear Volterra integral equation for |^(t,a). Its solution for 

op 

( 1 , 0 -) e | K+1 is 

r,o) dr (12) 

where K(t) is the total number of discontinuities in the slope 3x/8t or the control u 
in the time interval [t 0 ,t). In equation (12) the matrix $(t,t 0 ,a) satisfies the equations 

-^(t,t 0 ,a) = A(t,a) $(t,t Q ,a) (13) 

and 

^(Wo^) s 1 (14) 

Here, I is the identity matrix and $(t,t 0 ,a) is a (n+1) -square matrix function. Equa- 
tions (8) and (12) may be used to evaluate the total derivative of x(t,cr) with respect to a 
along the line t = tf(o). This result is 

m is t 

^-(a) = <f>^tf,Si,cr) Af^(a) -^(cr) + § 4?(tf,T,cr) B(r,a) -^(T,a) 

4 1 t n 


^(t,a) = 
8a 


dS, pt 

$(t,Si,a) Afi(a) -j2(o) + 4>(t,r,a) B(r,a) 


8U/ 

*8a 


+ 



where the notations 


x^(a) A x(tf(a) ,a) and u^(ct) A u(tf(a),a) 


have been used. 


(15) 



Each trajectory of the imbedding family must satisfy the terminal constraint 


j(x*,cr) se 0 

for all a. Hence variations in x^, given by equation (15), must satisfy 


(16) 


identically in cr. An expression for the required variation in terminal time tf can be 
obtained by substituting equation (15) into equation (16) and solving the resulting expres- 

dtf. . . . 

sion for -j— (cr). The result dtf/dcr can be inserted into equation (15) to obtain an expres- 

tion for dx^/d a, which is required to satisfy the differential constraint equation (16). 

Thus, one obtains r- 

K(tf) 

it = - S(x«,a) , /(/; uf ) T + c(x 1 ,u«,a) “ 

da 9cT vJXx* ,a) f(x f ,u f ) 


^ *( t f» s i» 0 ') Af i(°) ^(°) 
i=l 


+ J f *(tf,T,<y) B(r,a) |^T,a) dr 
tn 


(17) 


wherein 


c(xf,u f ,o) A I - 


f(x f ,u f )A^j(x f ,cx) 

vJ J (x f ,a)f(x f ,u f ) 


(18) 


With the definitions provided in the preceding paragraphs the field of solutions to 
the optimization problem may be shown to satisfy theorem 2. The proof of theorem 2 is 
contained in appendix B. 

Theorem 2 : If the stated optimal control problem possesses a field of solutions 
x(t,o) which are continuous and piecewise differentiable, along with the associated optimal 
controls u(t,o) which are piecewise continuous and piecewise differentiable, then x(t,o) 
and u(t,o) must satisfy the following three conditions: 

(1) Switching point condition - The functions Si(cr) which describe the intervals 
satisfy 

M T c(x f ,u f ,a) ^(tfjSijOr) A%(a) = 0 
for i = l,2,...,K(tf). 

(2) General point condition — Over any interval of finite duration in t, where a com- 
ponent Uj of the control function u(t,cr) is not on the boundary of the control space J2, 
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Mj(t,o) = pc(x f ,u f ,cr) $(t f ,t,a) B(t,a)j . = 0 


for t,a in Also if uj is on the boundary of the control space £2 then 
Mj(t,o) ^0 if u- = 0 !j 

Mj(t,o) i 0 if iij = /3j 


and 


f— 1 =0 if Mi(t,a) * 0 

Wj 1 


£ 

(3) Terminal control condition — The terminal control u (a) must minimize 
G]_(x^,u^,a) and the index of performance must satisfy 


(— \ (a) = min G^(x^,u^,ct) 

Wn+l uUn 


where 


G^x^u^a) £ - |J(x f ,o-) 


f n + l( xf ’ uf ) 


9<7 * V^'j(x^,a)f(x^,u f ) 

These three conditions are subject to the constraint that 
V^j(x f ,a) f(x f ,u f )< 0 


(19) 


Any practical application of theorem 2 involves a consideration of the singular 
situations where the terminal control u^ cannot be defined by the terminal control con- 
dition of theorem 2. The next section presents extensions to theorem 2 which must be 
considered in order to cope with singularities which arise in problem solutions. 

Extensions of the principal result .- As in algebraic problems there is no assurity 
that the imbedding process may be continued until the family of terminal manifolds col- 
lapses onto the one which is of interest. Indeed there are singular situations which arise 
during the application of terminal imbedding when the continuability of the process must 
be determined. The singularities mentioned above depend on the parameterization of the 
family of terminal manifolds. The singularities which can prevent continuation of the 
process can be divided into two categories: class A singularities which are due to the 
failure of the function Gi(x*,u f ,cr) of theorem 2 to define the terminal control u f and 
class B singularities due to the nonexistence of the derivative dx^/da of equation (17). 
These two classes of singularities are discussed, in order, in the following paragraphs. 
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By definition, class A singularities arise because of the failure of the terminal con- 
trol condition (theorem 2) to uniquely define the terminal control u^. These singularities 
occur when Gj( x^,u^,cr) becomes independent of some component of the vector u^. The 
terminal control condition requires u* to minimize the function G^(x^,u^,cr) subject to 
the constraints that u^ must be in ft and that V^j(x^,ct) f(x^,u^) < 0. 

Let fti be the set of controls u^ which satisfy this set of conditions. If this set 

f f 

does not define a unique u 1 the series representation of x^ + ^(a+e) is extended to 
include the second-order terms: 

x n+l( a+e ) = x n+l^ + G l( xf ’ uf ’°) e + G 2( xf > uf ’°) Y 

Then for u^ in ft-^ the quantity u^ must minimize G 2 (x^ ,u^ ,cr). Let ^2 be the set 
of controls u f which are in ft^ and minimize G 2 . If this set does not uniquely define 
a terminal control u^, then continue this procedure, considering higher-order terms to 
further restrict the set of terminal controls. An illustration of this procedure is pre- 
sented in the section "Applications to Nonlinear Analysis.” 

The class B singularities are defined by the zeros of the function vTj(x^,a) f(x^,u^) 
since at such points the derivative dxydcr cannot be defined according to equation (17). 
Consider u f to be determined as a function of the terminal state x f and cr, that is, 
u* = u f (xf,a) according to the terminal control condition of theorem 2. Define 

F(x*,o) A V^j(x f ,a) f(x f ,uf(x f ,a)) (20) 

The function F(x f ,a) vanishes for some x at any given value of a. The nature of this 
function is schematically illustrated in the upper sketch of figure 5. The shaded region 
represents a zone in x* space where f(x^,ct) is not defined. This zone will generally 
appear since there may be terminal states x* for which there is no control u^ directing 
the velocity vector f(x^,u^) into the terminal manifold for a given value of a. The 
point A in this sketch is a point for which there is no vector f(x^,uf) directed into the 
terminal manifold with u^ in ft. This is indicated by the vectorgram at point A repre- 
senting the set of vectors f(x*,wO with uf in ft. A similar vectorgram at point B 
shows that there are vectors f(x*,u*0 with u* in ft directed into the terminal manifold. 

Whether a class B singularity will result in termination of the process of evolution 
of the optimal trajectories depends on the behavior of dxfydcr at the singular point and 
on the manner in which the terminal manifolds evolve with respect to a. For example, 
if in a class B singularity the component of the vector dx^/da in the direction of 
V x j(xf,a) can be made equal to - -|^(x*,cr), then the evolution of the optimal trajectories 
may be continued. In order to do this, however, one may be required to add another dis- 
continuity into the field of optimal trajectories. The procedure for accomplishing this is 
illustrated in the section entitled "Applications of Terminal Imbedding." A singularity 
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Figurs 5*- Nature of class B singularities. 

for which the addition of another discontinuity to the imbedded family of control functions 
will allow continuation of the solutions is said to be a simple singularity. Sometimes it 
is not possible by the addition of discontinuities at the end of the trajectory to make the 
component of dx^/da in the V x j(x^,a) direction equal to - |i(x^,a). Then the singu- 
larity is said to be terminal since the evolution of optimal trajectories, with the param- 
eterization of terminal manifolds attempted, cannot be continued. The lower sketch of 
figure 5 illustrates a class B singularity for which a continuation of a continuous field of 
optimal trajectories is not possible. The problem considered is for the missile initially 
at point C to maneuver in such a way as to strike the target set indicated minimum time. 
For a c it is possible for the missile to strike the set of terminal conditions indicated 
in the sketch. However, at a = Oj the trajectory of the missile is just tangent to the set 
of terminal conditions indicated. For a > crj the missile must fly by the target set and 
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return to strike it. For the imbedding parameterization selected, the field of optimal 
trajectories is not continuous and by continuous imbedding it is not possible to continue 
the imbedding past a = o^. This does not mean that no solution exists; only that, for 
the imbedding parameterization selected, one cannot move continuously from a < a^ to 
cr = Oj > Oj in order to obtain the solution. 


For many problems the condition that the function jjJ- c(x^,u^,ct) <3?(tf,t,a) B(t,o) 
must vanish cannot be realized over an interval of finite measure in t. In these cases 
the general point condition of theorem 2 implies that 9u / da s 0 (almost everywhere) 
and the main points of concern are the switching point condition and the terminal control 
condition. The switching point condition is 

M T c(x f ,u f ,a) $(tf,S i5 a) Afi = 0 (21) 


The terminal state variation can be obtained from equation (15), for the special case 
where 9u/9a=0, as 




dSi 


da 


da 


da 


( 22 ) 


and the terminal constraints are 

j(x f ,a) = 0 (23) 

Equations (21) and (23) are algebraic relations between the variables tf, x*, 

S^, Sg;(tf) which constrain the integration of equation (22). As such they are dif- 

ficult to use in their algebraic form with equation (22) . Equations (21) and (23) are most 
easily used by applying a conversion to differential form similar to that previously devel- 
oped herein for algebraic systems. 

The differential form of the constraint equation (23) is 



Now consider equation (21). Differentiation of this equation with respect to a results 
in the expression: 
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cb£ 

da 


v x(M T c)(x f ,u f ,a) $(tf ,Si,a) A% + V u ( M T c)(x f ,u f ,a) $(t f ,S i5 a) Af^ 


rp 9C 
+ M 1 — 


8a 


(x f ,u f ,a) 4>(t f ,Si,a) Afj + /i T c(x f ,uf,a) |^(% s i> CT )^ 




( 25 ) 


for i = 1,2,...,K. In equation (25) the additional assumption that Af^ is constant has 
been used. From the properties of the transition matrix <i?(t,t 0 ,a), the following expres- 
sions are obtained: 


8$ 

8t 


(t f ,S i5 a) = A(tf ,a) *(t f ,Si,a) 


(26) 


_8£ 

atn 


(t f ,Si,a) = -$(t f ,S i5 a) A (S i5 a) 


(27) 


Evaluation of the matrix ^p(tf,Si,a) requires further consideration. First one 


notes that 


-2i(t,t 0 ,a) = — 
3cr ' ' 


8 a 


I+j* A(r,a) $(T,t 0 ,a) dT 

“tn 


thus, 


|f(t,to,o-) = f * If- (T.0-) ®(T,t 0 ,<7) dr + f ‘ A(r,o) f- (r,t 0 ;C r) dr 
l o l o 


This last equation is a linear integral equation for — (t,t 0 ,a) of the same form as equa- 

8a ' ' 

tion (9). Its solution is 


8$ 

8a 


(t,t 0 ,a) = J *(t,r,.a) — (r,a) $ (r,t 0 ,a) dr 
tn 


(28) 


which can be verified by direct substitution. Equations (26), (27), and (28) may be used 
in equation (25) to obtain 
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ifT 4T( t(jS . )CT ) v T( (J T c )( x f ;U f >0 .)^ + Af T 4 ,( tf ,S 1 ,<7)vT( M T c)(x £ ,u f ,o)^ 


+ c(x f ,u f ,a) 


dt 


dS; 


A(t f ,a) $(t f ,S i5 a) ^ - *(t f ,S i5 a) A( Si ,a) ^ 


da 


Af, 


K(t) 


t jjg. 

+ jLt T C(x f ,u f ,a) j^ f $(t f ,r,a) ^ Al(r,a) 4>(r,S i5 a) dr Af t —• 
= -M T -j^(x f ,u f ,a) $(t f ,S i} a) Af t 

for i = 1,2,...,K wherein the following relation has been used: 

K(t) 

|^t,a) = £ A^(t,a) 


k=l 


with 


A k (t,a) a 


~(t,o) *(t,S k> a) Af k , . . ^Mt,o) *(t,S k ,a) Af k 


(29) 


(30) 


Since the control space is such that ^ u^ ^ )3j for each component of the control vec- 


f „ 


tor with a?j ,/3j either finite or infinite then the control u- is either against a limit 
where du^/da = 0, or the function G^(x^,u^,a) has an internal minimum where 


9u* 


(x^u^u) = 0. In either event 

v^ T c)(xV,")^ = o 

since v u ( = |j£(x £ ,a) V u Gj. Hence, the final differential form of the switching 
point constraint equation (21) is; 


25 



Af^ # T (t f ,Si,a) ( M T c)(x f ,u f ,CT) + m T C(x f ,u f ,a) A(t f ,a) $(t f ,Si,a) Af t 


dS, 


M 


T C(x f ,u f ,cr) $(tf,S i5 a) A(Si,ct) Afi — i 


K(t) 


+ i u T C(x f ,u f ,a) J f $(t f ,r,a) ^ A^a) <f>(T,Si,a) dT A^ 


k=l 


dSj, 

da 


= - ^(M T c)(x f ,u f ,a) 4>(t f ,Si,a) Af t 


(31) 


which must hold for i = l,2,...,K(tf). 

Equations (22), (24), and (31) along with the terminal control condition of theorem 2 
form an implicit set of ordinary differential equations which are linear in the variables 

fj x f dtf dSi dS-tr 

— — , — — . At simple singularities the set of differential equations 

da’ da da da 

(eq. (24)) which was the condition used to define dt-f/da in the derivation of equation (17) 
for theorem 2 cannot be satisfied. By elimination of this equation and converting the 
remaining ordinary differential equations to equations with tf as the independent vari- 
able, instead of a, the behavior of the system of equations obtained by admitting another 
singularity to the field of solutions may be determined. Whether the singularity is simple 
or terminal may be verified by using this procedure which is illustrated by means of 
examples in the next section. 


APPLICATIONS OF TERMINAL IMBEDDING 


In this section, the concepts which were previously outlined are applied to problems 
in the aerospace sciences. An example of a standard optimization problem, illustrating 
the concepts of terminal imbedding for linear time invariant systems, is first presented. 
Then, a nonlinear singular trajectory optimization problem, using a simplified model of 
an airplane attacking a target vehicle, is solved. The purpose of these examples is to 
clarify the application of the various concepts which have been presented previously. 

Time -Optimal Control for Time -Invariant Linear Systems 
Consider the time optimal control of a linear system, described by 

x = Ax + Bu (32) 


26 



to the manifold described by j(x^) = 0 from an initial point x° with the control space 
O taken such that j Uj j = 1 for j = l,2,...,m. Applying the parameterization of the 
terminal manifolds, one obtains 

j(x^,a) = j(x^) - j(x°) + a (33) 

Here oj = j(x°), and cr is varied between 0 and Oj. The terminal control condition of 
theorem 2 for this example is 

min — (34) 

dCT u f € O V^j(x f ) [Ax f + Bu f ] 


The minimization here requires that 


u j = ■ s e n [ Bl ' v x J ( xf )]j 


,...,m where 



"i 

x > 0 

sgn x = < 

-l 

j 

x < 0 

! 

[undefined 

u 

o 


(35) 


Under the parameterization of the terminal manifolds used for this example, equa- 
tions (22) and (31) may be rewritten as 


dx£ 

dt f 


K (‘«) d S 

y h k — = Ax f + Bu f 

A kdt f 


(36) 


’ - ** J W Ahi ^ = vTj( x f) Ahi 

for i = 1,2,...,K, and equation (34) may be written as 


(37) 


da 

dt f 


where the norm definition 


B T V x j(x f ) | - vjj(x f ) Ax f 
n 

-II* 


(38) 


and the function definition 


h, A e A (‘r S l) 


B Am 


(39) 
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for i = have been used in equations (36), (37), and (38). Note that the class B 

singularities arising have been treated by a change of independent variable from a to tf. 
However, if the coefficient preceding dSi/dtf in equation (37) vanishes, then the equation 
will fail to define dSi/dtf and another type of singularity may be encountered. These have 
not been investigated since this problem has not yet occurred in application. 

To illustrate the application of this analysis, consider the problem of the undamped 
harmonic oscillator with a natural frequency of unity. The equation of motion of the 
oscillator is presumed to be 

x h + x h = u 

Now, let the control force u be bounded so that ( u | ^ 1. If the oscillator is disturbed 

from its equilibrium position, then consider the problem of applying the control force in 

2 2 ? 
such a way that x^ + x^ becomes less than some arbitrary r in the shortest possible 

time. 

Formulating this problem in state vector notation, the equation of motion is 

x = Ax + Bu (40) 

where 

x T = (x x ,x 2 ) = (x h ^ h ) 


and 



1 

0 


B T = (0,1) 


The equation for the terminal manifold can be written as 



Now for this problem the fundamental matrix 4?(t,0) = e^ is 
cos t sin t 

-sin t cos t 

and the gradient matrices to be used in equations (36), (37), and (38) are 
V x j(x f ) = x f 



(41) 


(42) 


(43) 


(44) 


(45) 
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and 


(46) 


V xx J (x‘) = I 

From equation (35) the terminal control is 

u f = -sgn Xg (47) 

Equations (41) to (47) may be substituted into equations (36), (37), and (38), along with 
equation (39) to construct a set of 3 + K(tf) ordinary differential equations in the 
3 + K(tf) dependent variables x^ jX^Sj, ...,£>£ and a. For the specific initial condi- 
tions (x 0 ) 1 = ( 2 , 2 ), figure 6 illustrates the evolution of the terminal time tf and 
switching points Sj(ct) and 82 (a) obtained by integration of the differential equations 
for this problem. In the range of values from a = 0 to cr ~ 0.6, K(tf) = 0; but at 
a « 0.6, the final state crosses the curve xi> = 0 in the x^x^ plane (fig. 7), an 
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Figure 7*- Time-optimal solution to problem of harmonic 
oscillator for a = 2. 

increase in K(tf) from 0 to 1 being necessary. From equation (47) for terminal states 

where Xg > 0 the terminal control is u^ = -1 and for terminal states with Xg < 0 the 

■f f 

terminal control is u 1 = 1. Hence, at values of a where the line x 9 = 0. As crossed 

by the terminal state, a discontinuity in the control function is clearly- indicated** 

the time at which the discontinuity occurs is tf which yields the initial condition * 

■>« £ * ... 

lim St^ , 1 ( a-. +A')=tf('a 1 '\ for cr-, corresponding to the singularity. The final state',: 

A — 0 v ' ' * ' $ v-i 'iv 

A>0 | ^ 

crosses the curve Xg = 0 again at a ~ 3.8, and again, it is necessary to increase the ' 

value of K(tf') from 1 to 2 so as to allow the process to continue. Also indicated in 

figure 6 are the controls in the sectors of the t,cr plane for 0 < t < Sj, SjAfc t'< Sij, 

and S 2 < t < tf. For this problem the value of a corresponding to any given r- ; can be 

obtained by using equations (33) and (43). This result is ’ " * j 



Figure 6 may be used to determine the control policy corresponding to time-optimal 
transition from the initial point (2,2) to the surface corresponding to a certain value of a. 
For example, to transfer from (2,2) to a circle with r = 2 with the trace of constant 
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a = 2, indicated by the dashed line in the t,a plane of figure 6, yields the time -optimal 
control policy indicated on the lower curve in figure 6. When this control time history 
is employed in equation (40), the phase plane trace presented in figure 7 is obtained. 


Applications to Nonlinear Analysis 

For an application of terminal imbedding to nonlinear problems, consider a sim- 
plified model of an aerial attack in a horizontal plane where one airplane A attacks a 
target T. The geometry of the situation is illustrated in figure 8. The target T is 



Figure 8.- Geometry of aerial attack. 


constrained to fly along the X-axis at a constant speed V-p, and, initially, the target is 
assumed to be at the origin of the coordinate system. The equations of motion for the 
airplane are 


X A 

= V cos 6 

^A 

= V sin 0 

6 = 

u c 

Ki — 

1 V 

V = 

K a(^- 





(48) 


In equations (48) , the variables x^ and y^ represent the position of the airplane A 
in the horizontal plane; 6 is the angle between the velocity vector and the X-axis, mea- 
sured counterclockwise, and V is the speed of the airplane. The controls of the airplane 
are u c corresponding to normal acceleration and V c corresponding to thrust. The 
constant Kp is a maximum normal acceleration so that u c ^ 1 and the constant K 2 
depends on the drag of the airplane. For the airplane selected for the study at a speed of 
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277 meters/second and at an altitude of 3658 meters above sea level, the constant K 2 
is approximately 4.54 x 10"^ meter - *. For a maximum normal acceleration of five times 
the acceleration of gravity, the constant Kj is approximately 49 meters/second 2 . 

These constants have been used in this example computation. 

Termination is assumed whenever the airplane maneuvers so that the target is 
within the circle indicated in figure 8. Mathematically, this termination condition is 
written in the form 


j( f) ( X A + R cos $ ~ x t) + (^A + R sin ~ 0 

2 

wherein the state vector (x*) T = (x^,y^,0*,V^,xlp) has been used. 

The method of terminal imbedding is used to solve the problem of completing the 
attack in the shortest possible time. The family of terminal manifolds consists of 

j(x*,o) = j(x f ) + (cr - l)j(x°) = 0 


This is equivalent to selecting a termination zone radius large enough to insure instant 
termination for a = 0 and gradually collapsing the radius until it becomes r f at a = 1. 
The radius of the termination zone corresponding to a value of a is given by 

r 2 (cr) = (r f ) 2 + (1 - a)j(x°) 

For this example r* has been chosen to be 30 meters. 

In order to apply the theory of terminal imbedding, the gradients of the terminal 
manifolds with respect to x* are required. These gradients are 


f f f 

x A + R cos 6 l - x T 


y A + R sin 6* 


7xj(x f ) 


R 

0 

-( 


f f 

y A cos 6 1 


^x A - xjpj sin 0 f 


x A + R cos 


0* - xQ 


(49) 
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and 



1 

0 

-R sin 0^ 

0 

-1 


0 

1 

R cos 0* 

0 

0 

Vxxj(x f ) = 

-R sin 0* 

R cos 0* 

-r|x^ sin 0* + ^x^ - xQ cos 0* 

0 

R sin 0-*- 


0 

0 

0 

0 

0 


-1 

0 

R sin 0^ 

0 

1 

✓ » 


For this problem the differential equations governing the transition matrix $(t,t 0 ,cr) can 
be integrated by quadrature. The transition matrix satisfies the equations 


-^-(t,t 0 ,a) = A(t,cr) $(t,t 0 ,a) 
^(Wocr) = I 


where the matrix A(t,a), as defined by equation (10), is 


0 0 -V sin 0 

0 0 V cos 0 


A(t,cr) 


0 0 


0 


0 0 0 
0 0 0 


cos 0 
sin 9 

-KjUc 

V 2 

-k 2 v 

0 


0 

0 

0 

0 

0_ 


(50) 


and B(t,o), as defined by equation (11), is 


B(t,o) 


0 

0 

^1 

V 

0 

0 


0 

0 

0 

K 2 

0 


(51) 
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where the control vector = ^u c ,V^j has been used. The dependence of all variables 

in equations (50) and (51) on t and a is understood. The matrix 4>(t,t 0 ,a) is then 


where 


1 0 y A (t Q ) - y A (t) <P U 0 



1 x A (t > - x A( t o) 
0 1 


024 0 

^34 0 


00 0 044 ° 

0 0 0 0 1 


0 44 = exp -K 2 




(52) 


(53) 



+ V(r) sin 0(t) exp^-K 2 JJ 1 V(r 2 ) dT 2 j dTj dr (56) 

In equations (52) to (56) the dependence of the functions on cr is also understood. 
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The terminal control condition requires that u* must be selected so that it mini- 
mizes the function Gj(x^,u f ,a) as defined in equation (19). For the time-optimal prob- 
lem, this is equivalent to maximizing V^j(x^) f(x^u^). From equations (48) and (49) the 
following equation is obtained: 

V^j(x f ) f(x f ,u f ) = a(x f ) + — y- j3(x f ) u* (57) 

x yi 

where 

a(x^) = (v* cos 6^ - Vrpj + R cos 6^ - + V* sin + R sin Q^j 


$(J) 


cos 0 1 - ^x^ - xiQ sin 


Hence, 


u* = sgn 


W* 1 )) 


(58) 


and is not defined when /3(x^) = 0. The speed control does not appear in equa- 
tion (57). The procedure of extending the series representation of p^x^a+e) to include 
second-order terms must be used to define the final speed control. To accomplish this, 
note that 

/ f f ) dG^x^u^a) 

Gpix 1 ,u 1 ,cr J / = — 


da 


Thus, 


G 2 (x f ,u f ,a) = 


[vjj(x f ) f (x f ,u f J) 


2-^f[ v x J ^ £ ( xl ’ uJ )] + n 


where the term y^ is independent of V^. Performing the indicated differentiation and 
using equation (58) gives 


G 2 (x f ,u f ,a) = 


[v^(x f ) f (xV)] 




+ y l 


where 


>(x f ) = 


X A “ X T + R cos cos ^ + ( y A + R sin sin ^ 


2K 1 R| i 3(x f )l 

(vf 


(59) 
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Since 


where y ^ 


€-44- 

(v f ) 2 +r 2 

is independent of 

V c’ 

fv • 

„ mm 

V = ( 

o> 

A 

O 

I 

[Vmax 

O 

A 

<o 


( 60 ) 


where V m i n is the minimum flying speed of the airplane and V max is the maximum 
speed of the airplane. 

Equations (58), (59), and (60) complete the definition of the final controls except 
for the singular control case. When ^(x f ) vanishes over an interval of finite measure 

-f 

in a, the terminal control u^ is not defined by equation (58). Instead, one should 
employ the general point condition of theorem 2 over the time interval from (tf-e ,t£J 
with e > 0. The general point condition for this problem implies that 


V^j(x f ) ^(tfjt^b^a) = 0 


where bj(t,a) is the first column of the B(t,o) matrix of equation (51). Substituting 
equations (49), (51), and (52) into the last equation results in 


- x^ + r cos e f )(y A - y A ) + (y A + R sin - x A ) = o (61) 

where the side condition /3(x f ) = 0 has been used. The last expression must hold over 
the time interval (tf-e,tfj if the control u^ is not on the boundary of the control space. 
Hence, differentiating equation (61) with respect to time t and dividing by Y gives 



Xrp + R cos 


^ cos 8 - 


y A 


6i ) 


+ R sin 8 1 ) sin 6 = 0 


(62) 


where the kinematic equations (48) have been used. Equation (62) is an identity in t 
over the interval [tf-e,tfj and hence differentiating it with respect to time yields 



+ R cos 0M sin 8 + 



+ R sin 


0 


cos 8 


K 1 U C 


= 0 


(63) 


Now, since equation (62) holds, the bracketed term of equation (63) cannot vanish. It fol- 
lows that the control u c in equation (63) must vanish. Hence, from the definition of u f 

as lim u(t,cr), the general point condition infers that u^ = 0 for pix*) = 0 over an 
t-tf c 

interval of finite measure in a. 
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Now that the terminal control has been determined, one may form the set of equa- 
tions (22), (24), and (31) for K = 0 to obtain 


dx 


f 




— — = V f cos 8^ 
dtf 


dy A f f 
— = V 1 sin 8 1 

dt f 


d^ _ K l u c 
dtf yf 


dV f 

dtf 



dx*, 

"dt^ 



(64) 


f f 

with u c defined according to equation (58) and V c defined according to equation (60). 

Equation (64) may be integrated until either = 0 or 6(x*) = 0. For the purpose 

of this example, it is assumed that /3(x*) vanishes first. This assumption is true for 

the example computation illustrated in figure 9. The initial conditions for the attack 

illustrated are 

f 

X A = "304.8 meters 


y^ = 3048 meters 
0 ° = 0 ° 

V° = 304.8 meters/second 
Xj, = 0 meter 


and the target speed is 304.8 meters/second. The maximum airplane speed is 
457.2 meters/second for this example. The value of a for which /3(x^) vanishes has 
been computed to be 0.89037. For this example, j3(x°) < 0 and 6(x°) > 0 sets the con 
trols in zone as u c = -1 and V c = V c max as illustrated in figure 10. 
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Terminal position of airplane, Xf,and target,x-j-f km 
Figure 9»- Terminal positions of both airplane and target. 
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For values of a greater than 0.89037, the region ^2 in figure 10 must be 
included in the analysis and equations (22) and (31) must be formulated for K = 1 with 
the controls in |2 being u c = 0 and V c = V Cjmax . This set of equations after con- 
siderable reduction for this example is 



dx 


f 


dy f . 


eU H" 3-n 

1 dt f z dt f 


A + a<, — + a 




dx^, 

dtf~ 


dSj 

+ a p- — ~ a R 
5 dt f 6 


J 


(65) 


where u^ = -1, u^ = 0, u^ = 0, and 


a l = y A " y A ' R sin 0f 


f 1 f 

a 2 = x A - x^ + R cos 9 X 


a 3 ~ “ R 


1 f / 1 f \ 
y A sin 0 + {x A - x A j si 


sin 9' 




a 4 = " a l 


4 


f f f 

a 5 = l x A “ X T + R C0S 9 




( X A 


n 


^y A + R sin 0 f 


vlccs^-( y i- y i)^fcS 


a e = V f 


( X A - 4) sin - 


f f 

y A cos e 


( 66 ) 
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and the notation x a = x a( S 1’ ct )» y A = y A( S l ,CT )’ = V ( S 1’ CT ) h ^s been used. When 

integrated, the set of equations (65) yields the portion of the terminal position graphs in 
figure 9 from a ^ to the final value of a of unity corresponding to a bomb radius of 
30.48 meters. The variation in terminal time and switching time obtained is indicated in 
figure 10. The optimal control time history is the trace on the t,a plane at <7=1. 

If, during the process of integrating equations (65) the function 6 (x*) had vanished, it 
would have been necessary to formulate equations (22) and (31) for K = 2 and introduce 
another region in the t,a plane. 

CONCLUDING REMARKS 

Imbedding theory has been applied in this report to variational or optimal control 
problems. This application was called terminal imbedding. Terminal imbedding involves 
imbedding the optimization problem of interest in a family of problems parameterized by 
the terminal conditions. By collapsing the terminal conditions of the family of problems 
onto those of the original problem while continuously modifying the control function 
appropriately the solution to the original trajectory optimization problem may be obtained. 
Using this method, necessary conditions have been derived analogous to those of the vari- 
ational calculus and the maximum principle of Pontryagin. This analogy is established 
in an appendix. In this work these conditions are called the switching point condition and 
the general point condition. An additional condition called the terminal control condition 
results from the terminal imbedding theory. This condition is not included in the maxi- 
mum principle since it is necessary only if the optimal trajectory is imbedded in a con- 
tinuous field of solutions; however, it is important in the construction of the field of 
optimal trajectories. With the use of the imbedding concept, the placement of singular 
subarcs in an optimal trajectory is determined. This is not easily accomplished by the 
maximum principle. 

An important consideration in the practical use of terminal imbedding is the inclu- 
sion of discontinuities in the field of optimal trajectories. These arise at terminal mani- 
folds where the continuability of the imbedding process is not assured or where the 
terminal control condition fails to define the terminal control. In applying the method of 
conversion to differential form for the solution of nonlinear algebraic equations, it was 
possible to derive a sufficiency condition which assures the continuability of the imbedding 
process (theorem 1). This condition has not been generalized to trajectory optimization 
problems and when obtaining solutions to these problems it is necessary to examine each 
singular situation as it arises to determine the continuability of the imbedding process. 
The nature of the singular points is that they may prevent continuation of the imbedding 
process or they may require the admission of another point of discontinuity in the field 
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An 

of optimal controls. For problems where — i(t,o) = 0 in regions of the t,a plane where 

d<y 

the control is free of discontinuities, the evolution of the points of discontinuity as a func- 
tion of the imbedding parameter a was obtained and was applied to a simplified model 
of an aerial attack as an illustrative example. 

Langley Research Center, 

National Aeronautics and Space Administration, 

Hampton, Va., February 24, 1971. 
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APPENDIX A 


ANALOGY WITH THE MAXIMUM PRINCIPLE 

Before demonstrating an analogy between the analysis presented in this report and 
the maximum principle of Pontryagin (ref. 4), a statement of the major results of the 
maximum principle as applied to the problem considered in this work is presented. Then 
some difficulties in applying the maximum principle to singular problems in control are 
discussed and the extraction of necessary conditions pertinent to singular control prob- 
lems from the maximum principle using the approach of Johnson and Gibson (ref. 7) is 
outlined. Finally, an analogy between the results of the maximum principle and those of 
this report is demonstrated. 

Presentation of the maximum principle is facilitated if a Hamiltonian function is 
first defined according to 

H(x,i//,u) A i// T f(x,u) 

where xjy is an (n+1) -dimensional vector sometimes called the costate vector. The state 
and costate vectors satisfy the canonical equations 

x = V^H(x,i//,u) = f(x,u) 

* = -V x H(x,!//,u) 

The basic results of the maximum principle are* that any control u which minimizes 
x n+ i(tf) must satisfy the following conditions: 

(1) It must maximize H(x,i//,u) over the entire trajectory 

(2) At termination H(x,^,u)|^_^ = 0 

(3) At termination the vector i/z^tf), the n-dimensional vector formed by the first 

n components of i//(tf), is normal to the terminal manifold and must be non- 

positive and can be taken as -1 

In the ideal application of the maximum principle, condition (1) is first used to 
define the optimal control in terms of the state and costate variables. That is, the optimal 
control u is related to x,^/ via the relation 

u = argmax H(x,i//,v) = u(x,i//) 
vefi 

This result, when substituted into the canonical equations, yields a complete set of 2n + 2 
ordinary differential equations 
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x = f(x,u(x,^)) 

P = -V x h(x,^/,u(x,!//)) 


APPENDIX A 


At this point, conditions (2) and (3) of the maximum principle are used to provide a com- 
plete set of boundary conditions for the last set of differential equations. These boundary 
conditions are 

ViM = - 1 

H(x,j//,u(x,^/))| t=tf = 0 
^ n (t f ) = kV x j(x n (t f )) 

where k is a scalar. These conditions together with the initial conditions x n (t Q ) = x° 
and x n+ ^(to) = 0 form 2n + 2 two-point boundary conditions for the problem and thereb 
reduce the optimization problem to a two-point boundary value problem. 

The terminology "singular control" refers to those special situations which arise in 
control theory where the idealized application of the maximum principle envisioned in the 
last paragraph cannot be realized. The basic difficulty is that the condition that u maxi- 
mizes H(x,i//,u) at every point of the trajectory may not be adequate in itself to define 
the optimal control; thus, the function u(x,i//) cannot be defined, making the conversion of 
the optimization problem into a two -point boundary value problem impossible. One of the 
problems in which this difficulty arises is the so-called "linear optimization problem" for 
bounded control where the Hamiltonian function is a linear function of a scalar control u 
and can be written as 

H(x>,u) =I(x,^) + uF(x,i//) 

Over any interval of finite duration where F(x,ip) vanishes along an optimal trajectory 
the condition that u maximizes H(x,i//,u) obviously fails to define u. Pontryagin, 
Boltyanskii, Gamkrelidze, and Mishchenko (ref. 4) have shown that the condition that 
H(x,i//,u)Jj._^ = 0 can be strengthened for autonomous free end-time problems to infer 

that H(x,i//,u) = 0 along the entire optimal trajectory. Johnson and Gibson (ref. 7) have 
employed this condition over the singular arcs to yield the identities 

F(x,^) = F(x,xj/) = F(x,xf/) = . . . = 0 

and 

I(x,i//) = I(x,i//) = I(x,^) = . . . = 0 
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which must be satisfied by any singular arc and can be used to determine the nature of 
the control along the singular arc. This procedure will be illustrated by a simple example. 

Consider the boat steering problem whose nomenclature is given in figure 11. 



Figure 11.- nomenclature for boat steering problem. 

The boat travels with unit speed with a turning capability limited to 1 radian per second 
and the following kinematic equations are descriptive of the motion of the boat: 

Xj = COS Xg 

Xg = sin Xg 
kg=U 

where the control u is such that j u j § 1. The problem considered here is to reach 
the shoreline (the Xj axis) in the shortest possible time. The Hamiltonian function is 

H(x,?//,u) =I(x,^) +uF(x,$) 

where 

I(x,l//) = ^ COS Xg + ^2 Sin Xg - 1 

F(x,i//) = t/'g 

The costate differential equations are 
V'l = P 2 = 0 

1^3 = sin Xg - cos x 3 
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According to the maximum principle the optimal control must satisfy the equation 
u = sgn provided that 0 over an interval of finite duration. If ’/'g = 0 over 

an interval of finite duration, then the relations 

F(x,i//) = 0 
F(x,^) = 0 
f(x,M = 0 


yield the results 
1^3 = 0 

sin X3 - ^ cos Xg = 0 

COS X3 + 

respectively, for and ^ nontrivial constants. The last two equations are con- 
sistent only for u = 0. The singular control can only be u^O, At this point it is noted 
that this is the isolated control action which is a straight course for the boat. All optimal 
voyages which are long enough will involve this singular arc and in many cases the singular 
arc will represent the major portion of the distance traveled. Therefore, construction of 
the singular arc cannot be discarded as a trivial problem. Optimal trajectories for this 
specific problem are segments of two types of trajectories: one represents full turning 
capability employed to direct the course of the boat perpendicular to the shoreline, and 
the other is the singular portion (u = 0), the straight line portion of the trajectories. 

To verify an analogy with the maximum principle, define the vector p{t,a) 
according to the relation 

<Mt,cr) = -$ T (t f ,t,CT)C T (x f ,u f ,cr) fj. 

where and C are defined by equations (13) and (18), respectively. Note that the 
vector i//(t,cr) satisfies the relations 

-^(t,o) = -A T (t,o) xp(t,a) 

Ol 

= -C T (x f ,u f ,a )/j. 

identically in a. 
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Now consider the function H A i/^(t,a) f(x,u). Write Hf A f (x^,u^). From 

the definition of Hf, 

Hf = -jlx T c(x*,u^,a) f(x f ,u f ) 

so that from the definition of c(x^,u^,a) 


H f = jii 


f(x f ,u f )v3'j(x f ,a) 


vTj( x f ,a)f(x f ,u f ) 


f(x f ,u f ) 


from which it immediately follows that 
H f = 0 

provided V x J (x* , ct) f (x^ ,u f ) ± 0 . 

At termination the vector i//(t,o) satisfies the expression 


i//(t f ,cr) = -C T (x f ,u f ,a)p. = -ju. 


+ 



vTj( x f,a) f(xf,uf) 



Note that the n+1 component of ^(tf,cr) is -1 and also that i// n ^tf,a) is normal to the 
terminal manifold generated by each value of a. 

Finally, note that V U H is given by the equation 

V U H = ^ T (t,o) V u f(x,u) = -/i T c(x f ,u f ,a) $(t f ,t,a) B(t,cx) 

Hence, the general point condition of theorem 2 implies that, for any admissible control 
variation, the component of V U H must be nonpositive and it follows that u(t,o) must 
maximize H since all admissible changes in u must not increase it. Any trajectory 
which satisfies the maximum principle with the costate variables taken at i//(t,cr) also 
satisfies the conditions of theorem 2 of this report. 
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PROOF OF THEOREM 2 


It is the purpose of this appendix to provide a rigorous justification of theorem 2 
of this report. Some equations given in the discussion of theorem 2 in the main body of 
this report are needed in the proof of the theorem; therefore, these equations are restated 
in this appendix. Because of the requirement that all admissible trajectories satisfy the 
governing differential equation (eq. (9)) the field of optional trajectories must satisfy the 
partial differential equation 

|^t,oj = f(x(t,o),u(t,o)) (8) 

on every interval hence, 

pt 

x(t,o) = x° + \ f(x(r,cr),u(T,cr)) dr 

to 


Because of the requirement that x(t,cr) and u(t,o) must be differentiable in both t 
and a on intervals ^ and that the functions Sj(o) be differentiable, it follows that 



(9) 


for t,a in In equation (9), K(t) represents the number of Sj(a) functions con- 

tained in the interval jjt 0 ,tj and the following definitions have been used: 


A(t,a) A V x f(x(t,a),u(t,a)) 

(10) 

B(t,a) A v u f(x(t,o),u(t,o)) 

(ID 


and 


Afi(o) ^ 


g im Q f ( x ( S i’ a )’ u ( S i“ e ’ CT )) " f ( x ( s i>°M S i +€ ’ a ))] 

e>0 


Equation (9) is a linear Volterra integral equation for -^(t,o). If the transformation 

9 a 

K<t> as- 

~(t,cr) = Af^(a) — ^(ct) 

o a Lj da 

1=1 


48 



APPENDIX B 


is used in equation (9) , the function x(t>°) satisfies the equation 

X(t,cr) = J A (r,a) x(r,ff) dr + J h(r,a) d r 
to to 

wherein 

K (t) dg 

h(t,cr) ^ B(t,cr) -^(t,a) + A(t,a) V Afj(a) —^(a) (Bl) 

9a Lj da 

i=l 


The unique solution for x is 

X(t,a) = f 4>(t,r,a) h(r,a) dT 

to 

which can be verified by direct substitution where 4>(t,t 0 ,a) satisfies the equations 

-^(t,t 0 ,CT)= A(t,a) $(t,t 0 ,a) (13) 

and 

S(t 0 ,t 0 ,tf) 3 I (14) 

Here, I is the identity matrix and <f>(t,t 0 ,cr) is a (n+1) -square matrix function. Using 
the definitions of x(t,°) and h(t, a), the solution of J|(t,a) can be obtained. After some 

reduction -^(t,a) , in turn, can be written as 
do 


K(t) 


^E(t 

9a 


,a) = ^ ^(t,Si,a) Afi(a) + J $(t,r,a) B(r,a) ~(T,a) dr (12) 

i=l ° t 0 


for all t,a in Equations (8) and (12) may be used to evaluate the total derivative 

of x(t,a) with respect to a along the line t = t f (a). This result is 


m 


i=l l o 


dT 


■ 5 U f ) ~“(o) 


da 


(15) 


where the notations x*(o) =x(tf(a),a) and u*(a) 4 u(tf(a),a) have been used. 
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Each trajectory of the imbedding family must satisfy the terminal constraint 
j(x^,ct) = 0 

for all a. Hence variations in x*, given by equation (22), must satisfy 



(16) 


identically in a. An expression for the required variation in terminal time tf can be 

obtained by substituting equation (15) into equation (16) and solving the resulting expres- 
dtf . 

sion for — (a). The result dtf/da can be inserted into equation (15) to obtain an 
da ' 

expression for dx^/da, which is required to satisfy the differential constraint equa- 
tion (16). Thus, one obtains 


da ’ ' 


9a 


fl 

(x f ,u f ) 


Vjj(: 

x f ,a) f 



c(x f ,u f ,a) 


( tf ) dg . 

^ $(tf,Si ,a) Afj(a) -j-±(a) 


i=l 


wherein 




f(x f ,u f ) V^j(x f ,a) 
vjj(x f ,a) f(x f ,u f ) 


(17) 


(18) 


An expression for the variation in the performance index x* + ^ can be obtained 

from equation (17). This is accomplished by premultiplying the equation by an 

(n+1) -dimensional unit vector ji whose n+1 component is unity. The poles of 

cbr dSi 

n+1 (a) serve to define the boundaries of the intervals, hence the functions -—-(a), 
da 1 


da 


cbr 


n+1 

For other regular points where --(a) is well defined and continuous one can use a 

da f 

Taylor's series expansion with a Lagrangian remainder to represent x n+ ^(a). Hence, 


x n+l (CT+e) =x --i ( ct) + 


dx 


n+1 


n+1' 


da 


M' 


(B2) 
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for e > 0 and some satisfying the condition a < cr^ < a + e . Equation (B2) holds 


dx 


so long as — - ~ + ^ (o) is continuous in the interval [a,a+e]. Hence, if x ‘n+l (0) isa 


do- 


minimum, for x* + ^(a+e) to be a minimum with respect to e then it is necessary that 


dx 


n+1 


^o^ be a minimum. Furthermore on considering the limiting form of equa- 


do v , f , f 

dx - dx i 

tion (B2) as e — 0 the continuity of ii(a) requires that —(a) be a minimum. 

do do 


dX 


f 

n+1 


From equation (17) it is seen that — (o) depends on dSi/do with i = l,2,...,K(tf), 


dx 1 


the function — i(t , a ) , and the terminal control u. The minimization of — ^i-(o) must, 
9 or do 

therefore, be taken with respect to this set of quantities. 

T dx*- 

Next, consider minimization of ju. — (o) with respect to the quantities mentioned 

da 

in the preceding paragraph. To do this one should consider the integral term of the form 



(B3) 


wherein M(t,a) and -^(t,a) are m-dimensional vector functions of t and a. The 

requirement of constraining u(t,a) to be in a manifold region £2 leads to directional 
constraints on -|^(t,a). In other words, at any given time t when u(t,a) is on a bound- 
ary of £2 a change in a at constant t must be consistent with the constraints and 

An 

hence — -(t,a) cannot be directed out of £2 at that boundary point. Also if u(t,a) is 
9(7 

not at a boundary point of £2 then there are no directional constraints and — (t,a) is 
totally unconstrained. 


Statement 1 : A necessary condition for -j^(t,a) to minimize is that over 

any interval of finite measure in t, say, £ where the ith component of u satisfies 
Q' i < u i c then the ith component of M(t,a) vanishes over £. 


Proof by contradiction: Assume that 9u/9a minimizes J^-^j and that «i< u^ < j3j_ 


over | when Mj(t,a) ^ 0. Select a comparison arc 9u/9a such that 
j + i and (|^j. s (fa)- exce Pt over the interval £ where 


I 


for 



(t,o) s - eMi(t,a) 


with e > 0. The value of the functional J(~*] evaluated along this comparison arc is 
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8u 
• 8a 


= J 


8u 

3a 


- e f M?(t,a) dt 
J £ i 


so that J jjBu/ 9oj < J (j8u/ 8aJ. This contradicts the assumption that 8u/ 3a minimizes 
j£*J. Hence, Mj(t,a) = 0 over the interval £. 

Statement 2 : A necessary condition for 8u/ 8 a to minimize j[j J is that over 
any interval of finite measure in t, say, £ where Uj s /3j over £, Mj(t,a) S 0 

and if Mi(t,a) < 0 then ^|^ (t,a) = 0. 

Proof by contradiction: First, the condition that Mi(t,a) 0 over £ is proved. 
Assume that over £, Mi(t,a) > 0 and that |^(t,a) minimizes The constraint 

that u^ = /3. implies that (t,o) ^ 0 over £. Select then a comparison curve 

.8^ (t,0) ■ (toV 1 ’ 0 * £or !#1 “ d (to)/*’ 0 * ■ d$j <M except 

the interval £ where ( — )(t,o) = [— ) (t,cr) - e for some e > 0. This arc satis- 

\ 9o/i \ ocr/i 

fies the directional constraints. The value of the functional jf-^j evaluated along the 


4^t,a) such that 

3(7 


over 


comparison arc satisfies 


8u 

9ct 


= J 


8u 

8a 


* e J M i (t,a) 


dt 


Hence, if Mj(t,a) > 0 over £ then J [au/saj < J [au/8a] which contradicts the assump- 


tion that -^(t,cr) minimizes j[^. From this contradiction the conclusion that Mi = 0 
over £ is obtained. 

Next the fact that if Mi(t,a) < 0 over £ then 0^(t,a) = 0 over £ is proved. 
Assume that 8u/8a minimizes and that over £, Mi(t,a) < 0 with 0^.(t,a) =£ o. 

The directional constraints on -j^(t,o) require that g 0. Hence only < 0 


over £ need be considered. If 


such that 


|^j < 0 over £ construct a comparison arc 8u/8a 
| he f — ] for i # i and ( s ( \ except over the interval £ where 

j V 8cr/j \9a/i V^A 


~(t,o) Q |^(t,a) + e < 0 for e > 0 small enough. The value of the functional j(jJ 
evaluated along the comparison arc satisfies 


8u 
8 a 


= J 


8u 

8a 


+ e 


J Mj(t,a) 


dt 
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and if M^(t,a) < 0 over | then J [^9u/acrj < J |^9u/3crj which contradicts the assumption 
that 3u/3a minimizes J(jJ. Hence -2^t,cr) must vanish over any interval £ in t 
where Mj(t,o) > 0. 

Statement 3 : A necessary condition for 3u/3cr to minimize is that over 

any interval of finite measure in t, say, £ where Ujs ffj over £, Mj(t,a) I 0 and 

if M^cr) > 0 then (^ (t,o) = 0. 

Proof: The proof of statement 3 is similar to that of statement 2 and is not 
presented. 


Statement 4: A necessary condition for 


J [3u/ 3cr] = 0. 




to minimize 




is that 


Proof: The proof of statement 4 follows directly from statements 1, 2, and 3. If, 
over any interval a component of the control vector, say u i? is not on the boundary 
of the control space then Mj(t,a) = 0 from statement 1. If the component is on the 
boundary of the control space and if Mj(t,o) =£ 0 from statements 2 and 3, 3u/3ct must 
vanish. Hence over any interval £ of finite measure in t the integrand MT(t,cr) J^(t,o) 

of j[^ must vanish if •^^t,o) is to minimize jQ^j. Therefore if -^i(t,cr) minimizes 

then j[3u/3(t] = 0. 


Statements 1 to 4 provide the basic tools for minimizing the integral term of 
f 

p T Applying statement 4 with M T (t,cr) taken as ju. T c(x*,u f ,a) <t>(t£,t,o) B(t,o), 

T dx^ 

the minimization of ju -—(a) is seen to require that 

da 


M' 


c(x f ,u f ,a) J f $(t f ,t,a) B(t,a) dt = 0 

t 0 


(B4) 


dS, 


Now consider the remaining terms. Note that the functions -^(a), i = 1,...,K, are 
totally unconstrained. Hence the coefficients preceding each of the terms dSi/dcr in 

rn dx^ 

/i 1 -j-j(cr) must vanish independently so that any discontinuities must occur at times 
Si(o) satisfying 


M T c(x f ,u f ,n) $(tf,Si,a) Af^o) = 0 


(B5) 


Equations (B4) and (B5) may be used to determine the variation in the performance 

f 

index x ^(o) after substitution into equation (17) to obtain 
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M 


T fL = Gi( x f , u V)A-M( x V) 


n+l 


(xV) 


9a' 


vjj(x f ,a) f(x f ,u f ) 


(B6) 


From the definition of an admissible trajectory, J(x,o) > 0 for x evaluated along an 
admissible trajectory and J(x,cr) = 0 only at termination. Hence, 9J/8t^0 at termina 
tion and this implies that 


v3’j(x f ,a)f(x f ,u f )^0 


For a given parameterization, equation (B6) serves to define 


dx 


n+l 


da 


■(a) as long as there 


■* 4+1 

da 


is an admissible control u^ satisfying vjj(x^,a) f(x^,u^) < 0. Since 
minimized it follows that u^ minimizes Gj(x^,u^,a) subject to the constraint 

V^j(x f ,a) f(x f ,u f ) < 0. 


(a) is to be 


At every point t,a the control must either be on the boundary or in the interior 
of the control space £2. If over any interval of finite measure in t a component uj of 
the control vector is not on the boundary of £2, statements 2 and 3 imply that 


Mi(t,a) = [p T C(x f ,u f ,a) $(t f ,t,a) B(t,a^. = 0 


Now, if over any interval of finite duration in t a component u^ of the control vector 
is at the boundary of the control space £2, statements 2 and 3 infer that 


and 


Mj(t,a) ^ 0 if Uj = aj 

Mj(t,a) SO if Uj = jSj 



(t,a) = 0 


if Mj(t,a) * 0 


The results of the preceding paragraphs can be compiled into theorem 2 of the text. 
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